DOCUMENT RESUME 



ED 265 165 



TM 850 496 



AUTHOR 
TITLE 

INSTITUTION 
SPOHS AGENCY 



PUB DATE 
NOTE 



PUB TYPE 



EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



ABSTRACT 



Vaughan, Eva D.; And Others 

Factors Associated with Differences in Patterns of 
Program Implementation: A Three Year Study. 
Pittsburgh Univ., Pa. Learning Research and 
Development Center. 

National Inst, of Education (ED), Washington, DC; 
Special Education Programs (ED/OSERS), Washington, 
DC. 

Apr 85 

29p.; Paper presented at the Annual Meeting of the 
American Educational Research Association (69th, 
Chicago, IL, March 31-April 4, 1985). 
Speeches/Conference Papers (150) — Reports - 
Research/Technical (143) 

MF01/PC02 Plus Postage. 

Analysis of Variance; Correlation; ^Educational 
Assessment; Educational Environment; ^Educational 
Innovation; Elementary Education; Elementary School 
Teachers; Item Analysis; Longitudinal Studies; 
^Models; Program Effectiveness; Program Evaluation; 
^Program Implementation; *Staff Develoj^ent; 
^Teaching Skills 

^Adaptive Learning Environments Model; Project Follow 
Through 



Purposes of this study were: (1) to examine patterns 
of implementation of an innovative educational program with a 
built-in staff development component over a period of three years; 
(2) to compare patterns of implementation of the Adaptive Learning 
Environments Model's (ALEM) three types of teaching skills, including 
generic, adaptive, and program specific; and (3) to examine 
site-to-site differences in patterns of program implementation over 
time. Data for the study were obtained from classrooms in four school 
districts which implemented the ALEM as the core instructional 
program in grades K through 3, in conjunction with participation in 
the National Follow Through Program. Sites varied in ethnocultural, 
socioeconnmic, and geographic characteristics. Teachers' classroom 
implementation was measured by administration of the Implementation 
Assessment Battery for Adaptive Instruction (lABAI). Results, 
obtained using analyses of variance, item analysis, and correlational 
analysis, were summarized to show few marked differences in patterns 
of change over time for the three types of teaching skills. When 
ample support was provided, improvements in implementation were 
likely to result; xae absence of or inadequacy of support, however, 
lead to stagnation or decrement of teaching skills. (LMO) 
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Factors associated vlth Differences 
In Patterns of Program Implementation: 
A Three Year Study 

The successful Implementation of an Innovative educational program in a school 
is a difficult task, requiring commitment and effort on the part of administrators 
and teachers alike o In a study of federally funded projects, Perman & McLaughlin 
(1977) found that over half the innovative programs were either aborted soon after 
adoption or discontinued after a relatively short time. District- and school-level 
administrative support, especially by the school principal, was crucial for 
successful implementation. However, the bottom line is thj classroom teacher, who 
determines whether and to what degree the innovative practice is actually put in 
place. 

Research on school change has identified a number of factors which are related 
to teachers' willingness to implement an innovative program. These factors include 
clarity of the program's goals and methods, congruence of these goals and methods 
with teachers' goals, teachers' perception of their own efficacy and that of the 
program for bringing about important outcomes in students, and the provision of 
appropriate training (Baker & Showers, 1984; Berman & McLaughlin, 1978; Doyle & 
Ponder, 1977-1978; McLaughlin & Marsh, 1979; Showers, 1983). 

Even if all of these factors are present, successful classroom implementation 
of an innovative program is not an all-or-none phenomenon, especially if the program 
is complex and requires wide-ranging changes in teacher behavior. Hall and his 
collaborators (Hall & Loucks, 1979; Hall, Loucks, Rutherford > & Newlove, 1975) 
propose that teachers go through a sequence of stages of concern about and use of an 
innovation, progressing gradually from concerns with the mechanics of "how to do it" 
to impact-oriented concerns, and from piecemeal application of components to their 
integrated use. Wang and her colleagues have documented changes in degree of 
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Implementation of an Innovative adaptive Instruction program over the course of a 
school year and showed that changes In Implementation were closely related to the 
type and amount of training support available to the teachers (Wang 6f Gennarl, Ipp?. 
Wang, Vaughan, 6f Dytman, 1985). 



Moreover, these kinds of long-term changes are likely to Interact wi.th the 
cyclical nature of the school year. Studies of students' standardized achievement 
test scores h ^e consistently found an Increase In average achievement from fall to 
spring, followed by a decrease over the summer from spring to^ fall (/Anderson, 1984; 
Gordon, 1984). Since teachers, like students, must readjust to school routines 
after the long summer vacation, and, In most cases, adjust to a new set of students. 
It is likely that a similar effect occurs in measures of teacher performance. 

Despite considerable evidence that long-term changes in implementation of 
innovative programs are to be expected, especially in programs that include 
systematic, on-going training support, few studies have measured such changes over 
more than one year. Therefore, a major goal of this study was to examine patterns 
of Implementation of an innovative educational program with a built-in staff 
development component over a period of three years, as well as from fall to spring 
within each year, and from each spring to the following fall. 

The program involved in the study is the Adaptive Learning Environments Model 
(ALEM), an educational program designed to make instructional provisions that are 
adaptive to student differences in regular classes (Wang, 1980). The ALEM has been 
implemented in a variety of settings as a general education program, a malnstreamlng 
program for mildly handicapped students, and/or an alternative model for 
compensatory education programs such as the National Follow Through Program and 
Chapter I. The effectiveness of the ALEM in serving diverse groups of students has 
been demonstrated in terms of Improvement in classroom processes and achievement 
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gains (Wang, Gennarl, & Waxman, 1985; Wang, Peverly, & Randolph, 1984; Wang & 
Walberg, 1983). In addition, the ALEM has been validated by the Joint Dissemination 
Review Panel of the Department of Education ( Educational Programs That Work , 1980). 

Since 1980, the first year of data collection for this study, the ALEM has 
Incorporated a data-based staff development component. This component includes 
procedures for monitoring and measuring degree of program implementation, then using 
the results to design and deliver individualized training to teachers. Detailed 
descriptions of the staff development component and evidence of its effectiveness 
may be found In Wang & Gennarl (1983) and Wang, Vaughan, 6f Dytman (1985). 

The ALEM, like any other complex innovative program, includes features which 
vary in "newness" for teachers. Many of the teaching skills required for ALEM 
implementation are ones common to effective teaching in any setting, e.g., 
explaining, demonstrating, motivating. We refer to these skills as generic teaching 
skills. Still other skills required by the ALEM are common to all adaptive 
instruction programs, though not (or to a lesser degree) to conventional teaching, 
^•g»> prescribing individual assignments to students based on diagnosis of their 
learning needs. We have called these skills adaptive teaching skills. Finally, the 
ALEM requires certain unique teaching skills, e.g., use of a management system 
called the Student Self-Scheduling System. These kinds of skills may be called 
program-specific teaching skills. Even excellent teachers in traditional programs, 
i.e., teachers with high levels of generic skills, are likely to experience 
difficulty in learning and using new (adaptive or program-specific) technioues. 
Moreover, Howey (1983) has suggested that, unless teachers are supported In the use 
of such techniques, they are likely to revert to older more familiar ones. Due 
factors such as these, pattern of program implementation may vary from one type of 
skill to another. A second purpose of this study, therefore, was to compare 
patterns of implementation of the ALEM's three types of teaching skills: generic, 
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adaptive, and program-specific. 

Finally, the Rand studies of school change showed that Innovative programs were 
more likely to be adopted and continued by schools If their Implementation was 
adapted to meet the needs of the particular site. Conseauently , It seemed 
reasonable to expect variations In ALEM Implementation from one site to another, A 
third purpose of this study, therefore, was to examine site-to-slte differences In 
patterns of program Implementation over time and for each of the ^Mree types of 
teaching skill. 

In summary, the research questions addressed by this study were: 

1. To what degree and In what ways does Implementation of the ALFM vary over 
a period of three years and, within years, from fall to winter or spring? 

2. How do levels and patterns of implementation differ for generic, 
adaptive, and program-specific teaching skills? 

3. How do patterns of implementation vary from one site to another? 

Method and Procedure 

Data S ource 

The data for this study were obtained from classrooms in four school districts 
(hereafter referred to as Sites A, B, C, and D) which implemented the ALEM as the 
core instructional program in Grades K through 3 in conjunction with the districts' 
participation in the National Follow Through Program. While selected components of 
the ALEM had been implemented in some of the classes for nearly 10 years, systematic 
Incorporation of all the program features began in September, 1979, one year prior 
to the first year of the present study. 



The sites varied in ethno-cultural , socioeconomic, and geographic 
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characteristics. Site A is located in a large, raidwestern city, where the ALEM was 
implemented in two inner-city schools. Site B is part of a small, white, midwestern 
farming community in which all three schools implemented the ALEM. Site C is in a 
large, sparsely populated, mainly white, rural Appalachian county where the ALEM was 
implemented in three schools. Site D is located in a medium-sized city where the 
ALEM was implemented in three racially mixed schools. 

Two criteria were used to select classrooms (i.e., teachers) for the study. 
The first was continuous implementation of the ALEM in the teacher's class over the 
1980-81, 1981-82, and 1982-83 school years. The second criterion was availability 
of data from at least two implementation assessments for each year. These criteria 
were met by seven teachers at Site A, 11 teachers at Site B, 10 teachers at Site C, 
and 14 teachers at Site D, yielding a total sample of 42 teachers. All but one of 
the teachers had had two or more years of experience in implementing the ALEM prior 
to 1980-81. 

Instrumentation 



Teachers' classroom implementation was measured by administration of a battery 
designed to measure performance indicators of skills considered critical to 
effective implementation of the ALEM, the Implementation Assessment Battery for 
Adaptive Instruction (lABAI). The battery includes classroom observation fonns for 
direct assessment of teacher and student behavior, checklists for assessment of 
classroom design, instructional materials, and classroom records, and student and 
teacher interviews for assessment o2 less observable program features (e.g. 
teachers' knowledge of student learning styles). For additional information aboot 
the battery's content, administration, and validation, see Wang, Catalano, & Gromoll 
(1983). 
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The number of performance indicators (items) in the battery was 96 in the form 
used in 1980-81 and 1981-82; nine items were added in a ] 982-83 revision for a total 
of 105 performance indicators. Tn order to assess implementation of generic, 
adaptive, and program-specific teaching skills, the 105 performance Indicators were 
independently sorted by two trained raters into sets corresponding to the three 
skills categories. The initial percentage of agreement between the raters was 71 
percent; differences were resolved by discussion. Of the 105 items, 49 were 
identified as generic, 32 as adaptive, and 24 as program-specific. 

Each performance indicator (item) yields a score of pass (1) or fail (0). 
Teachers' scores in each skill category were determined by adding across the items 
in that category. A total score, the sum of all item scores, was also calculated 
for each teacher. Since the number of items per skill category varied, raw scores 
were converted to percentages to permit comparisons. 

Design and Analysis 

Trained site personnel administered the battery at least twice in each of the 
three years of the study. All sites collected data in the fall of each year (the 
fall round). However, some sites collected degree-of-implementation data again in 
winter only, some in spring only, and others in both winter and spring. Tn order to 
provide comparable data for all sites across all three years, winter and spring 
scores were combined and referred to as the "winter/spring round." The combination 
is justified by the fact that studies of ALEM implementation have consistently found 
few differences between winter and spring scores (Wang, 1981, 1982, ]983a, ]98?b, 
1984). 

Analysis of variance was used to assess differences in degree of implementation 
as a function of year (3 levels), round (2 levels), and site (4 levels). Separate 
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analyses were run for generic, adaptive, program-specific, and total scores. 
Infonnation about differences in levels and patterns of implementation of the three 
types of teaching skills was obtained by comparing the results of the analyses. Tn 
addition, item analyses of the 1980-81 and 1982-83 data sets were performed to 
identify teaching skills which were particularly difficult for teachers to implement 
in one or both of those years. 

Correlational analyses were conducted in order to assess consistency of 
teaching skills within individual teachers. To measure consistency over time, all 
intercorrelations of the same skills at two different points in time were computed. 
To assess consistency among the three skill categories, correlations among generic, 
adaptive, and program-specific scores were computed at each measurement point. For 
the correlational analyses, scores of teachers at each site were first converted to 
standard scores in order to control for differences in site means and variances. 
The level of significance for all analyses was set at .05. 

Results and Discussion 



The results of the analyses of variance, the item analyses, and the 
correlational analyses are discussed below in relation to the study's three research 
questions. 

Variations in Implementation over Time 

The results of the analyses of variance of total, generic, adaptive, and 
program-specific scores are summarized in Table 1. Changes over time are indicated 
by the Year, Round, and Year x Round effects. The nature and direction of changes 
over time are displayed in Figure 1, which shows the percentage mean level of 
implementation of indicators of each type of teaching skill in fall and In 
winter/spring of each year. 

ER?C 9 . 
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The Year effect was significant in the analyses of adaptive and 
program-specific but not generic skills. The figure indicates that this effect 
reflects an increase in adaptive and program-specific means over the years, while 
generic means remained approximately the sama. The Pound effect, significant in all 
analyses, appears in the figure as the expected cyclical effect: Tn all three skill 
categories, mean levels of implementation increased from fall to winter/spring 
within each year, then decreased between winter/spring and the following fall. 
However, the cyclical effect decreased over the three years, as indicated by the 
significant Year x Round effect in all analyses. Figure ] shows that, in each year, 
implementation of all types of skills began at a higher mean level than the previous 
fall, then tended to reach approximately the same level in winter/spring. 

The fact that the mean level of adaptive and program-specific, but not generic, 
skills increased over the years is not surprising. Studies have shown that the 
complex skills which contribute to effective teaching in all educational settings ~ 
skills such as explaining, questioning, and demonstrating — are not likely to 
Improve over time in experienced teachers without intensive training, e.g., coaching 
(Joyce & Showers, 198?; Rosenshine, 1983; Showers, 1983). On the other hand, 
provision of systematic, data-based staff development activities, incorporated in 
the ALEM from fall, 1980, can be expected to — and did — improve implementation of 
the program's relatively novel features. The finding of significant changes in 
levels of implementation of adaptive and program-specific skills, even in teachers 
with considerable experience in implementing the ALEM, provides support for Pall's 
model of teachers' long-term, gradual adaptation to a new program. 

The cyclical Round effect was present in all skill categories, thus 
demonstrating that both familiar and novel teaching skills tended to "drop" in fall 
and to require a certain amount of time to become re-established. Perhaps this is 
not so surprising; at the start of a school year, both teachers and students must 
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readjust their behavior after a long vacation, and must learn to accommodate to one 
another and to the classroom rules. Such major adjustments could well disrupt the 
exercise of even highly practiced teaching skills. After the teacher and class have 
settled into a routine, by lata fall or early winter, the skills can be expected to 
return to previous, or even higher, levels. 

Perhaps the clearest demonstration of the effectiveness of systematic, 
data-based staff development is the decrease in the cyclical winter/spring-to-fall 
"drop" over the years in all skill categories. Though the mean level of 
implementation reached in winter/spring remained approximately the same, the 
data-based training support provided to teachers, seemed to enable them to get the 
program underway progressively more quickly each fall. 

Despite systematic changes in mean level of implementation over time, few of 
the correlations computed between measures of the same skill at different points in 
time were significantly different from zero, especially in the case of adaptive and 
program-specific skills. That is, individual teachers who, compared with others at 
the same site, were relatively "high" (or "low") in a given skill category at one 
measurement period did not necessarily remain "high" (or "low") at another. This 
result is probably due to the individualized nature of data-based staff development 
in the ALEM, which provides training to each teacher based on his/her current needs. 
If, for example, one teacher received training in adaptive skills while another 
received training in program-specific skills, the teachers' relative performance 
levels in the two skill categories might be expected to change. 

Patterns of Implementation as £ Function 
of Type of Teaching Skill 

Certain similarities and differences in patterns of implementation of generic, 
adaptive, and program-specific skills over time were noted and discussed in the 

ER?C 11 



10 



previous section. The similarities and differences can be summarized as foil 



ows: 



1. Generic skills differed from adaptive and program-specific ones in that 
the mean level of the latter but r,ot the former increased significantly 
over the three years of the study. 

2. The three types of teaching skills exhibited a similar cyclical effect, 
consisting of an increase in mean level of implementation from fall to 
winter/spring t^ithin each year, followed by a decrease from winter/sprine 
to the next fall. 

3. The three types of teachin.(t skills were similar in that they all showed a 
decreased cyclical effect over the three years. 

Tn summary, few marked differences were found in patterns of change over time for 
the three types of teaching skill. 



When mean levels of implementation are examined, however, a large difference is 
evident between generic and adaptive skills, on the one hand, and program-specific 
skills, on the other. The overall mean levels of implementation of ger.srlc and 
adaptive indicators were both greater than 90 per cent (92.4%, and 93.4%, 
respectively). Moreover, as Figure 1 shows, generic and adaptive means were very 
similar at all measurement points. Though generic mean scores slightly exceeded 
adaptive means in i980-81, while adaptive means exceeded generic mean scores in 
1981-82 and 1982-83, these differences were small ~ usually less than two 
percentage points. 

The overall percentage mean of the program-specific scores, on the other hand, 
was only 80.] percent. Moreover, program-specific means remained consistently lower 
over all three years than the means of the other two skill categories. Although 
they approached nearer to the generic and adaptive means in winter/spring of each 
year than in fall, they never "caught up": The difference between the 
program-specific means and the generic and adaptive means was always greater than 5 
percentage points and usually 10 percentage points or more. 
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The item analyses conducted on the 1980-81 and 1982-83 data sets throw 
additional light on the difference in mean level of implementation of 
program-specific teaching skills, on the one hand, and generic and adaptive skills, 
on the other. Based on an arbitrary criterion of 40% or more failures in a given 
year as a cutoff point between "difficult" and "easy" items, 22 items were 
identified as difficult in 1980-81. Sixteen (73%) of these items were 
program-specific performance indicators, while only 6 (27%) were adaptive or 
generic. Of the 16 originally difficult program-snecific items, 7 Mm remained 
difficult in 1982-83, while 9 (56%) became easier. Of the 6 non-program-specific 
difficult items (4 generic and 2 adaptive) in 1980-81, only 1 (17%) remained 
difficult in 1982-83, while the other 5 (83%) became easier. Thus, program-specific 
items were not only more likely to be difficult to implement in the first place, but 
were also less likely to become easier over the next two years. 

The high levels of implementation of the relatively novel adaptive teaching 
skills by the second year ~ as high as those of the more familial generic skills 
and considerably higher than the program-specific means ~ was, at first, 
surprising. However, it can probably be explained by the presence in the yVLEM, of a 
highly structured management/support system for the diagnostic-prescriptive 
component of the educational program. The support system includes especially 
prepared, individualized basic skills curricula, explicit rules regarding use of 
tests and other diagnostic technicues, and detailed prescription sheets which list 
multiple activities related to each curricular objective. These materials and rules 
make, both the goals and methods of the program's adaptive teaching behaviors 
explicit and clear. Berman & McLaughlin (1977) concluded that innovative practices 
with clear goals and objectives are more likely to be adopted and continued hy 
teachers; it is likely that the clarity of methods and procedures greatly 
facilitated teachers' implementation of adaptive skills. 
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Another reason for the relatively high level of Implementation of adaptive 
skills may be that such skills are probably not as "new" as they may at first 
appear. Though extensive individualization of instruction is unlikely in 
traditional settings, good traditional teachers have always adapted their 
instruction to some degree to individual differences among students* Good 
traditional teachers notice when a student has difficulty and try to figure out why; 
when a student fails to learn, they vary their teaching styles or methods to meet 
his/her needs; they often group students for instruction, especially in reading^ 
based on the students' current performance levels. Thus, though the ALEM and other 
adaptive instruction programs include more comprehensive and more structured methods 
for adapting instruction to student needs, the basic concepts and techniaues of 
adaptation are rather widely known and practiced. 

The ALEM's program-specific skills, on the other hand, are not only complex and 
novel; in addition, they lack a highly structured management/support system, such as 
that available for adaptive skills. Program-specific skills reauired for effective 
implementation of the ALEM include the orchestration and appropriate integration of 
instruction based on the program's prescriptive curricula with the use of 
exploratory learning centers and student self-scheduling. The appropriate use of 
exploratory learning centers requires that teachers develop a wide range of learning 
options, activities for which teachers often have little time. Because 
self-scheduling is new not just to teachers but also to students, teachers must 
teach students to use the system while learning it themselves. Program-specific 
skills did improve markedly over time, probably due to the ALEM's built-in training 
support co.nponent. However, given their novelty, complexity, and lack of structural 
support, it is not surprising that they remained more difficult for teachers to 
acquire and maintain than either generic or adaptive skills. 

Despite the large differences in level of implementation of program-specific as 
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compared with generic and adaptive teaching skills, the correlations among all three 
types of teaching skills in Individual teachers at any given time were 
overwhelmingly positive and significant. That is, an increase (or decrease) of a 
teacher's score in one skill category was likely to be associated with an increase 
(or decrease) in the others as well. This is probably due, at least in part, to the 
fact that skills in all three categories are part of a single, integrated 
instructional system, wherein a change in any aspect affects the other aspects as 
well. Teachers' motivational characteristics may also be a factor. For example, a 
teacher who is motivated to work hard to implement an innovative educational program 
is likely to show positive changes in a variety of teaching skills, while a teacher 
lacking such motivation may show little or no change (or a negative change) in all 
of them . 

Site-to-Site Differences in Patterns 
of Implementation 



As Table 1 indicates, the Site effect was highly significant in all analyses. 
That is, the mean level of implementation of all three types of teaching skills 
differed significantly from one site to another. However, the direction of the 
differences varied among the skills categories, as revealed In Table ?, which 
displays site means and relative ranks in each skill category. The table shows 
that, with the exception of Site C, whose mean level of implementation was highest 
in all three skill categories, no other site was consistently highest or lowest 
compared with the others. For example, Sito B had relatively high generic and 
adaptive means, but ranked lowest in implementation of program-specif xc skills; Site 
D, on the other hand, had the lowest generic mean, but outperformed both Sites A and 
B in implementation of program-specific skills. 



The Site x Year interaction was signif leant In all analyses, indicating that 
year-to-year changes differed from one site to another. Figures 2, 3, and 4 show 
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patterns of Implementation over time as a function of site for generic, adaptive, 
and program-specific skills, respectively • Examination of the figures shows that 
each site showed a unique pattern of change, which was fairly consistent over all 
skill categories. For example. Site A showed little change in level of 
implementation of any category from 1980-81 to 1981-82, followed by increased levels 
in all categories from 1981-82 to 1983; whereas Site B showed an increase in all 
categories from 1980-81 to 1981-82, but no change (generic and adaptive) or a 
decrease (program-specific) from 1981-82 to 1982-83. 

Comparisons of Figures 2, 3, ami A reveals another interesting difference in 
sites' patterns of implementation of generic, adaptive, and program-specific skills. 
Whereas all sites started at about the same (relatively low) level on generic and 
adaptive implementation indicators in fall, 1980, and then became progressively more 
different, the sites differed markedly in the degree of implementation of 
program-specific performance indicators from the start. 

These site differences and patterns of change are explainable in terms of the 
amount and type of available training support at the four sites. It will be 
recalled that little systematic, data-based training had been conducted prior to 
1980-81, but that such training became an integral part of the ALEM's implementation 
at the four sites after that point. Thus, the uniform levels of generic teaching 
skills in fall, 1980, probably reflect the fact that teachers at all sites came from 
comparable types of preservice training programs and had bad comparable 
post-training experience. Similarly, the uniform initial degree of Implementation 
of adaptive skills can be explained by the similarity in strategies and materials 
used across the sites for individualizing instruction. Program-specific teaching 
skills, on the other hand, are based neither on traditional training and experience 
nor on concrete, structural supports. Therefore, their degree of implementation in 
fall, 1980, probably depended largely on the extent to which teachers were 
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encouraged or helped to use the new techniques at that time. Such encouragement and 
help might well have varied from one site to another. While school administrators 
and support personnel at certain sites may have emphasized the centrality of 
program-specific skills, those at other sites may have perceived these skills to be 
peripheral (or, at least, secondary in importance to other components), resulting in 
marked site differences in level of implementation of program-specific skills. 

The presence of systematic, data-based training as an ongoing program component 
over the next three years can be used to explain both the general increase in mean 
levels of implementation at all sites and also the increasing differences among 
sites. In the ALEM, data-based training is based on the results of implementation 
assessment. Given the results, training and support personnel at each site must 
decide on training priorities. It is highly probable that different sites would 
select different program components for trainirg emphasis; differences in training, 
in turn, would produce different levels of implementation and patterns of change. 

Some support for this explanation was obtained by a re-examination of training 
data collected during a study of the ALEM's data-based training program (Wang & 
Gennari, 1983). In that study, data on number of hours of training devoted to the 
ALEM's critical features during one year (1981-82) were collected and aggregated 
over ten sites, including those of the present study. Re-calculation of hours of 
training within eacli of this study's four sites revealed certain relations between 
training emphases and levels of implementation. For example. Rite D, whose teachers 
had relatively high mean implementation scores on program-specific skills, was found 
to have devoted relatively more training time to program-specific skills, compared 
with generic and adaptive skills, than the other three sites; Sites P and C, the 
sites with the highest mean level of implementation of adaptive skills, devoted 
relatively more training time to these skills, compared with generic and 
program-specific ones, than did Sites A and D. 
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Since the 1981-82 training data were aggregated over all teachers at the sites 
(including, but not limited to, the 42 teachers in the current study) and were 
collected during only one year, the data cannot be regarded as conclusive. However, 
they provide some support for the contention that site differences in patterns of 
implementation are at least partially accounted for by site differences in type and 
amount of training support. 

At any rate, it is clear that, despite the fact that all sites were 
implementing a single educational program having well-defined critical features, the 
methods and patterns of implementation varied markedly. These findings provide not 
only additional support but also quantitative evidence for the Fand study conclusion 
that, as innovative programs are continued and maintained at a given site, they are 
modified and adapted to meet the needs of the particular institutional setting. 

Conclusion 

The study's finding of varying patterns of implementation of an innovative 
educational program over time as a function of type of teaching skills, sites, and 
teachers within sites provides convincing evidence that program implementation is 
not an all-or-none phenomenon. Even in a well-established program, taught by 
experienced teachers, level of implementation is likely to fluctuate. Though some 
part of this fluctuation is explainable by the cyclical nature of the school year, 
other factors such as type or amount of training and other types of support are 
probably much more powerful. The results of the study suggest that, when ample 
support is provided, improvements in implementation are likely to result: the 
absence of or inadequate levels of such support, however, may lead to stagnation or 
even decrements in teaching skills. 

Both the patterns of change in mean level of Implementation over time and the 




differences among sites argue for the centrality of carefully designed and delivered 
staff development in support of program implementation. For maximal effectiveness, 
such staff development should be based on assessed needs of individual teachers* In 
addition, since innovative programs tend to include many components and since 
training time is limited, support personnel need to give considerable thought to 
developing training priorities consistent with the needs of the school and the 
district. In particular, more intensive training than is usually provided is 
probably necessary to bring the level of complex, novel skills (such as the 
program-specific skills of the present study) to a high level. Finally, since 
implementation of innovative programs usually requires extra effort on the part of 
teachers, support personnel need to attend to motivational factors as well as skill 
development, in order to increase teachers'^ willingness to put forth the extra 
needed effort. 
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Tabic I 



Source of 
Varijnce 

Site 

Year 

Round 

Site X Year 

Silc X Round 

Year X Round 

Sile X Year x Round 



^"r?lf .ARA??'' °^ Sites, Years and Rounds 

for Total lABAI Scores and Scores for Generic, Adaptive and Specific Catego, '.s 



Degrees of 
FrcedoD 

3 

2 

1 

6 

3 

2 

6 



Total 



Note: Error sources of varience:iyere: i-i 
for sile: irte-^teachers'^witK^ d.f. 



Generic 



Adaptive 



for year and site x year: year x teachers within sites with 76i f 
k; lound and sile x round: round x teachers within sites with 38 d f 
for year x .ound and site x yearx rouo-l: year x round x teachers within sites for 7^d f 



*N.S. - Not Significant 



Specific 



F 


P 


F 


P 


F 


P 


F 


P 


3.96 


.05 


19.11 


.0001 ; 


11.65 


.0001 , 


6.23 


.005 


10.40 


.0005 


2.73 


N.S.* 


16.49 


.0001 ; 
j 

.0001 1 


11.42 


.0001 


118.19 


.0001 


16.94 


.0005 


28.17 


120.77 


.0001 


10.71 


.0001 


7.59 


.0001 


3.70 


.005 


12.10 


.0001 


4.62 


.01 


1.89 


N.S. 


0.27 


N.S. 


6.64 


.005 


8.01 


.001 


3.21 


.05 


1 

4.47 


.05 


7.57 


.005 


2.78 


.05 


0.72 


N.S. 


1.34 


N.S. 


6.57 


.0001 
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Table 2 



Mean Level of Implementation of 
Generic, Adaptive, and Program-Specific 
Skills as a Function of Site 



Level of Implementation 



Generic Adaptive Program-Specific 

^^ean Percent Rank Mean Percent Rank Mean Percent Rank 



94.2 


3 


90.5 


4 


76.5 


3 


95.0 


2 


95.8 


2 


75.9 


4 


95.1 


1 


96.1 


i 


83.5 


1 


P7.4 


4 


91.0 


3 


82.8 


2 
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f h 
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. • Specific' 



^ Figure 1 Percentage mem of Gerierlc. Adaptive, and Specific lABAI score.; for all sites combined across three years of imp!emer,tation. ' 
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